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1. INTRODUCTION 

As technology grows, methods of communication such as remote collaboration in telepresence which 
allows communication between distant user has increased the expectations and demands in various fields [1]. 
Even though they are physically apart, people can feel present with the use of telecommunications technology. 
Telepresence is a set of telecommunication technology that allows users to present at a remote location through 
a virtual projection. The concept appears to be captured by the compound construction of "tele" and "presence" [2] 
in which “tele” refers to telecommunication [3], and “presence” is a sense of being in the virtual environment [4]. 
Thus, telepresence is a suitable method for people to collaborate remotely as it is very advantageous and 
provides better visualization. Marvin Minsky from [5], one of the first to use the phrase "telepresence" to 
describe the sensation of being far away while using a teleoperator, is where the concept of telepresence 
originates. Numerous methods have been developed to enable telepresence since Marvin Minsky from [6] first 
expressed the theory. 

The advancement of telepresence technology has changed the way people communicate and connect 
with each other. One of the advancements in telepresence types is video conference (VC). VC is a type of 
telepresence that provides high-definition video and stereo-quality audio and experience face-to-face 
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interaction with people at a distance [7]. However, VC applications such as Skype, Webex, and Zoom are 
constrained to two-dimensional (2D) display. Jo et al. [8] VC consists of a fixed camera view, which is only 
able to show the users’ upper bodies, thus resulting in consistent mutual gazes and limited room for movement. 
It is unable to convey such non-verbal communication and is far from giving co-presence sense to the users. 

Hence, in order to improve the user experience with telepresence [9], has claimed that life-size 
telepresence are ables to facilitates more natural interaction since people can fully see each other and improve 
non-verbal communication such as gaze, body language and gestures. There is a good possibility that social 
interactions, such as authority or dominance and persuasiveness, will be more evenly divided and natural if the 
user is displayed at life-size [10]. Therefore, it is one of the potential benefits of having the user displayed in 
life-size. In order to produce a life-size telepresence, most researchers are using head-mounted display (HMD) 
devices to project the user [11]-[15]. Despite that, the use of HMD devices can be cumbersome as it limits 
face-to-face communications [16]. The HMD has a limited capacity for gaze input due to its small display 
optics, which are positioned in front of each individual eye causing a restricted field of view (FOV). In addition 
to this, using HMDs can be an extremely unpleasant experience for participants. Park et al. [17] binocular 
display systems such as HMD show a separate image to the left and right eye, which can cause users to 
experience discomfort and motion sickness. 

Presented by Pejsa et al. [18] is aRoom2Room: the system makes use of projected augmented reality 
to enable life-size interaction between two remote participants while they are both physically present. The 
projection of life-size virtual replicas of distant participants takes place in a distant location. Participants in the 
system are not required to wear any wearable body equipment. It enables the participants to move around freely 
and interact with one another from various angles while maintaining accurate perspectives. Next, produces a 
life-size holographic telepresence using a projector, transparent glass and holographic film [19], [20]. Before 
sending the visuals and sound over the internet, holographic telepresence systems, according to [19], collect 
and compress images of actual, distant individuals or surrounding objects. After transmission, it decompresses 
the pictures before projecting them at the distant site. The potential of holographic telepresence to deliver a real- 
time, full-motion 3D projection of a remote user makes it the next technological step in communication [21]. 

In this research, a projector is utilized to produce a holographic life-size telepresence that can replace 
the conventional VC telepresence system. By utilising long throw projectors that emit uniform luminosity, it 
is able to project a life-size image of a person the at remote location. To track user posture in full body 
movement, the tracking required is discussed in the next section, together with the networking and holographic 
setup. The result of this research paper will enable an efficient communication for long-distance users using 
life-size telepresence and consequently able to save the transportation cost for travelling. 


2. METHOD 

This section explains the methodology of the proposed method to stimulate the holographic 
projection-based for life-size telepresence. To achieve this, there are several phases which include the full- 
body capture, followed by networking and holographic setup. The holographic setup involves at the local user 
workspace area. These phases are further explained in the subsection below. 


2.1. Full-body capture 

The tracking technology is an essential part of this research to ensure that life-size telepresence can 
be achieved. In this research, the Kinect v2 sensor is used to track information retrieved about the user's full- 
body posture. Microsoft Kinect, also referred to as a skeleton tracker, is a low-cost depth sensor that works 
with the Microsoft software development kit [22]. It enables intuitive verbal and gestural interaction between 
people and computers and gaming consoles without the need for extra devices [23]. Resolution, precision, body 
tracking, and resistance to shifting lighting conditions were only a few of the drawbacks of the first iteration 
of Kinect (Kinect v1). The second iteration of Kinect, or Kinect v2, considerably improved upon these 
drawbacks [24]. 

The placement of a Kinect sensor is significantly influencing the accuracy in measuring the full body 
of the user. The height of the Kinect sensor is set at 1.2 meters from the ground surface and the distance is set 
at 2.1 meters. Dziedzic et al. [25], the recommended height range to capture human body position is between 
0.6 meters and 1.8 meters from the ground, while the distance range is between 0.5 meters and 4.5 meters [26]. 
The user needs to be situated in a rest position such as seating or standing and facing the Kinect sensor for body 
tracking.Rather than colour, the Kinect sensor retrieves depth data for the 3-dimensional position. Figure 1| 
illustrates the tracking technique process using a Kinect sensor. After frame validation, the depth information 
that is retrieved by the Kinect sensor is capable of identifying a human figure by using a large database known 
as skeletal data and their accompanying depth pictures. The skeleton data coordinate mapping to the user body, 
with the colour frame superimposed on top of it all. Lastly, the results are displayed in real-time. 
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Figure 1. Tracking technique process using Kinect V2 sensor 


2.2. Networking 

As telepresence required more than one user to participate, it required a network connection to invite 
remote users [27]. As a result, the networking needed for this research is described in this subsection. This 
study uses the Unity framework and the Agora.io package to create a network connection and enable real-time 
communication. Real-time engagement (RTE) platform Agora.io is well renowned for enabling developers to 
construct robust in-app experiences. Software defined real-time network (SD-RTN™) developed by Agora is 
the world's most commonly utilized and intelligent real-time communication (RTC) network. It offers global 
coverage, with the most comprehensive coverage available anywhere on the planet. Users from different 
locations can be connected to one room using Agora.io. 

It is necessary to obtain a token from the application server to verify the user's identity before gaining 
access to the application in Agora.io. This allows the user to make use of the network connection. Both local 
user and remote user from different locations will be able to create and join the room as soon as the application 
server has validated the token. Figure 2 displays the flowchart that describes the implementation of Agora.io's 
networking. After the Agora.io services have been activated, the user will proceed to the lobby to enter the 
room number as shown in Figure 2. It is necessary for the local user to generate a room identification (ID) 
before a remote user may be invited to join the room. Both the local user and the remote user will be able to 
connect and display holographic as soon as the remote user has obtained the room ID and entered it. If the 
remote user enters a different room ID, the remote user won not be able to connect to the local user. 
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Figure 2. Networking structure for telepresence 
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2.3. Holographic setup 

The holographic setup is important in this research as it influences the size of the holographic display. 
Providing the life-size holographic is necessary for this research to improve user experience using telepresence. 
The holographic setup is based on [19] research. Luevano et al. [19] conduct research which address the 
effectiveness of distance learning between the students and the professor by using holographic telepresence at 
the University of Tecnolégico de Monterrey. The results of the research achieve the life-size holographic 
telepresence by using long throw projection. 

The holographic setup requires a large space with a computer, an internet connection for live 
transmission of the data captured, a projector to emit the projection and audio equipment. To set up, up the 
hardware, the distance between the projector and the wall is crucial as it determines the size of the holographic 
display. To achieve the life-size display, the projector requires a larger distance between the wall and the 
projector. The setup is shown in Figure 3. Where in Figure 3, the distance between the projector and the wall 
is fixed at 3.84 meters. The projector is tilted at estimation angle of 50 degrees, until the projection is not 
overreached to the ground. The local user is located next to the projector, and the computer is situated behind 
the user in this configuration. 


Wall 


Local user 


Computer 


3.84 meter 


Figure 3. Holographic setup for life-size telepresence 


3. RESULTS AND DISCUSSION 

In this section, the results of the research is presented together with the comprehensive discussion. It 
consists of the remote user workspace, full-body capture, user interface, local user workspace and evaluation 
results. As this research aims to produce a life-size telepresence by using a projector-based holographic display, 
a stable internet connection is a requirement to execute the application and the human tracking device is a 
mandatory, a Kinect sensor. 

Figure 4 demonstrates the remote user Figure 4(a) workspace area and Figure 4(b) the results of the 
full-body capture. According to Figure 4(a), the remote user is standing in front of the Kinect sensor while 
wearing headphones for audio communication. The Kinect is placed using a tripod to ensure its FOV can 
capture the remote user’s full body for tracking. Meanwhile, Figure 4(b) shows the results of the tracking 
technique proposed in this research using Kinect sensor. The tracking technique can capture the remote user’s 
full body movement in real time. The background of the remote user also has been removed for the remote user 
to appear in telepresence at the local user workspace. 


(a) 


Figure 4. The remote user (a) workspace area and (b) results of the full-body capture 
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As explained in the previous section, remote and local users are required to enter room ID to use the 
Agora.io service. Therefore, the user interface is designed for both users to enter the room ID number. The user 
interface is illustrated in Figure 5. In Figure 5, an input column for the room ID and two buttons for join and 
create are provided. The create button is for the local user to create a new room ID number. Meanwhile the join 
button is for the remote user to join the room that has been created by the local user. If the remote user join the 
room with a different ID, the remote user will not be able to get connected to the same room as the local user. 


LIFE-SIZE for 


TELEPRESENCE SPACE 


Figure 5. User interface 


Once the remote user enters the same room as the local user, an immediate connection will be established 
between the two users. Through the holographic display, the remote user will have the appearance of being in the 
workspace of the local user. Figure 6 shows the local user Figure 6(a) workspace area and Figure 6(b) the output 
of the remote user in holographic display. In Figure 6(a) the local user workspace includes a projector, a laptop 
computer, pieces of audio equipment, and headphones. The local user will be able to communicate with the 
remote user through its holographic display from a different location. The life-size of the remote user’s virtual 
copy makes the local user feel the presence of the remote user. Meanwhile, Figure 6(b) displays the holographic 
representation of the remote user at a relatively close range. 


(a) 


Figure 6. Local user (a) workspace area and (b) the holographic display of the remote user 


To evaluate the life-size telepresence using the holographic application, an evaluation that includes 
participation of 14 respondents has been conducted to gather respondents’ feedback through questionnaire. All 
the questions were presented using a 5-points Likert scale, where 1 was the strongly disagree and 5 was the 
strongly agree. The respondent's feedback is analyzed and plotted in graph as shown in Figure 7. In Figure 7(a), 
71.4% (10) of respondents have strongly agreed that the partner has appeared in a life-size, whereas 28.6% (4) 
of respondents rated agreed that the partner has appeared in a life-size. This demonstrates that this research has 
achieved life-size telepresence. Additionally, the respondent also can communicate with their partner in a fluid 
and efficient way because of the life-size telepresence. More than half of the respondents (10) completely 
agreed that they could interact with their partner in a fluid and efficient way. While 28.6% (4) agreed they 
interact with their partner in a fluid and efficient way. The respondent bar chart shown in Figure 7(b), supports 
that the communication effectiveness in telepresence can be achieved due to the life-size display. 
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Figure 7. Respondent’s feedback on (a) the life-size telepresence and (b) interactions fluid and efficient 


Next, Figure 8 shows the graph of respondent feedback on the Figure 8(a) presenceness of the partner 
and Figure 8(b) network delay while using the application. In Figure 8(a), 28.6% (4) of respondents agree it 
was as if their partner was in the room with them. Meanwhile, the remaining 71.4% (10) of respondents strongly 
agreed that their partner was present in the room. This demonstrates that the respondents were aware of their 
partner's presence in the room. For the network delay during the communication in Figure 8(b), 35.7% (5) of 
respondents strongly agree that there is no network delay during using the application, while 28.6% (4) agree 
that there is no network delay when using the application. However, 14.3% (2) strongly disagreed, 7.1% (1) 
score disagree and 14.3% (2) neither disputed that they experienced network delay while using the applications. 
The results on the network delay may be variable according to the time during user use the application. The 
network may be delayed during peak hour time due to high demands and volume usage. 

Lastly, Figure 9 shows the respondents Figure 9(a) comfort and enjoyment using the application and Figure 
9(b) overall satisfaction with the application. According to the graph in Figure 9(a), 85.7% (12) of respondents 
strongly agreed that the application was comfortable and pleasurable to use. Another 14.3% (2) agreed that the 
application was pleasant and enjoyable. Figure 9(b) shows that 85.7% of the respondent rated strongly agree that the 
application give overall satisfactory. Meanwhile, 14.3% (2) of the respondent rated agree. These high percentages 
of positive responses clearly indicate that the application has been well received and has met the expectations of the 
users. Overall, the graph of the respondents’ feedback on the application shows a positive outcome. 
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Figure 8. Respondent’s feedback on the (a) presenceness of the partner and (b) application’s network delay 
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Figure 9. Respondent’s feedback in application’s (a) comfortability and enjoyment and (b) overall 
satisfaction 
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4. CONCLUSION 

In a conclusion, this research has successfully achieved its aim and objective to produce life-size 
telepresence using the holographic display. It can project remote users at a remote location through a virtual 
projection in real-time for collaboration. The virtual projection of the remote user is not only limited to the 
head and shoulders but offers a full-body remote user on a life-size scale. Indirectly, able to support non-verbal 
communication including body language, gestures, posture and gaze. It is also able to save the cost of travelling 
and a large monitor display. The use of holographic displays in this research provides freedom to the user. The 
user does not require to wear any wearable device or contact a wired device that can restrict the user’s 
movement. Users are free to move and able to have efficient communication with remote users through long- 
distance communication. This is somehow able to increase the user experience using telepresence. 

Furthermore, this research can be associated with several suggestions and improvements to enhance 
its efficiency, flexibility, and reliability. One of the suggestions for future improvement is to explore the 
implementation of life-size telepresence for a mixed reality (MR) environment or a virtual reality (VR) 
environment. Besides that, this research can be used as a comparative study for user experiences using HMD 
and without HMD. User interaction also can be explored to provide an appropriate interaction method for 
telepresence. 
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